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Abstract 

For the inclusion problem involving two maximal monotone operators, under 
the metric subregularity of the composite operator, we derive the linear conver¬ 
gence of the generalized proximal point algorithm and several splitting algorithms, 
which include the over-relaxed forward-backward splitting algorithm, the generalized 
Douglas-Rachford splitting algorithm and Davis’ three-operator splitting algorithm. 
To the best of our knowledge, this linear convergence condition is weaker than the 
existing ones that almost all require the strong monotonicity of the composite oper¬ 
ator. Withal, we give some sufficient conditions to ensure the metric subregularity 
of the composite operator. At last, the preliminary numerical performances on some 
toy examples support the theoretical results. 

Keywords: Linear convergence, metric subregularity, generalized PPA, over-relaxed 
FBS algorithm, generalized DRS algorithm, three-operator splitting algorithm 


1 Introduction 

Let X,Y and Z be the finite dimensional linear spaces endowed with the inner product 
(•, •) and its induced norm || • ||. Given the maximal monotone operators A, R: Z ^ Z and 
"d-cocoercive operator C: Z ^ Z, we focus on the composite operator inclusion problem 

0 G with T:=A + B + C. (1) 

We are interested in the case that one of A, B is single valued and Lipschitz continuous. 
Unless otherwise stated, we always assume that 0 for problems (1). 

The inclusion problem (1) has many applications such as the variational inequality 
problem [25], the problem of finding a common point of closed convex sets [3, 4] and covers 
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many classes of convex optimization problems. Specifically, we consider the following 
unconstrained nonsmooth composite convex minimization problem with the form that 

min f{x) + g{Vx) + h{x) (2) 

where /: Z —>■ [— 00 , + 00 ] and g:Y —[— 00 , + 00 ] are low semicontinuous convex functions, 
5 : Z —> (— 00 , + 00 ) is continuous differentiable and gradient Lipschitz convex function 
and P : Z —>■ Y is linear operators. Involved in the dual variable y G Y, it is easy to 
check that solving the optimization problem ( 2 ) is equivalent to solve the inclusion 

(0, 0) G (yl + S + C)(x, y) (3) 

with C{x,y) = (V/i(x) , 0) being the cocoercive operator, A{x, y) = {T>*y, —Vx) being 
the single valued, Lipschitz continuous operator and B{x,y) = [df{x),dg*{y)'j being the 
maximal monotone operator where dg* is the conjugate function of g, which indicates that 
the composite convex problem ( 2 ) can be reformulated as a special case of ( 1 ) with the 
above specified operators A,B,C. Furthermore, when g is also continuous differentiable 
and gradient Lipschitz, the problem (2) can be directly represented as the form of (1) 
with A = df, B = V* o Vy o D and C = V/i. Additionally, the inclusion problem ( 1 ) 
is highly related to the linearly constrained two-block separable convex minimization 
problem which has many applications as listed in [ 6 ] and takes the following form that 

s.t. Qu -b T-Lv = c. 

In addition, it has a dual problem falling into framework of (2) with the following form 

min cf>(z) := r{g*z) + g*{'H*z) - (c, z), 

z£L 

which also can be reformulated as a special case of inclusion (1). Here /: X —>■ (— 00 , -|-oo] 
and y: Y —>■ (— 00 , -t- 00 ] are closed proper convex functions whose conjugate function are 
written as f* and g*, respectively, f? : X —)■ Z and 7^ : Y—^Z are linear operators whose 
adjoint are g* and Ti*, respectively, and c G Z is a vector. 

Let bTiyT" denote the resolvent of an operator T of index 7 > 0 , i.e., = {Z. 

If by chance the calculation of is easy, then the classical proximal point algorithm 
[36] or its over-relaxed version (the generalized PPA [19]) is a desirable solver for the 
inclusion problem (1). However, in practice the estimation of is usually much more 
difficult than that of b/lyyi, J-yS and Zf^c- Motivated by this, Davis and Yin [16] proposed 
a three operator splitting method for the inclusion (1) by using and bTlyg. Moreover, 
when one of B and C vanishes, several two operator splitting algorithms, including the 
forward-backward splitting (FBS) algorithm [34, 22, 10, 11], the Peaceman-Rachford 
splitting (PRS) algorithm [30, 35], and the Douglas-Rachford splitting (DRS) algorithm 
[30, 18], have been developed by using or/and bTlyg, J^yC- The convergence of these 
splitting algorithms have been well studied. In view of the strong assumption required 
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by the linear convergence, some authors recently focus on the iteration complexity of 
these algorithms [26, 14, 29]. By contrast, the study for their convergence rate is quite 
deficient. To the best of our knowledge, several existing linear convergence rate results 
(see [30, 10, 23, 15, 16]) all require the strong monotonicity of one of the operators A, B 
and C and single valued and Lipschitz continuous property of B. It is well known that the 
strong monotonicity assumption of A,B or C is too stringent. Recently, Liang et al. [29] 
and Bauschke et al. [5] establish the local linear convergence rate of substantial splitting 
algorithms based on the Krasnosel’skh-Mann fixed point iteration[27, 32] scheme with 
the metric subregularity assumption and the (bounded) linear regularity assumption on 
the fixed point operator at a point of its graph, respectively. The condition used by 
Liang et al. [29] is shown to be equivalent to the metric subregularity condition of at a 
point (a;*,0) G gph for the generalized PPA algorithm and over-relaxed PBS algorithm 
according to lemma 3.3 in the following. 

The main contribution of this paper is to derive the linear convergence rate of the 
above several splitting algorithms and Davis’ splitting algorithm [16] under the metric 
subregularity of the operator at a point (x, 0) of its graph. In addition, as will be 
shown in the section 3, the metric subregularity of an operator at a point of its graph is 
weaker than some existing regularization conditions such as the strongly monotone and 
the projective type error bound [40] on the fixed point operator. 

2 Preliminaries 

This section recalls some necessary concepts and lemmas that will be used in the subse¬ 
quent analysis. Firstly, we introduce some concepts associated to an operator T: X ^ Y, 
for which we make no difference from its graph gphT := {{x,y) G X x Y | y G T(x)}. 
The domain and range of an operator T; X ^ Y are respectively dehned as 

domT = {x G X ] Tx 7 ^ 0} and ranT = {y G Y | G X such that (x,y) G T}. 

The inverse of T is given by T~^ := {(y, x) G Y x X ] (x, y) G T}. For any c G M, we let 
cT = {(x,cy) ] (x,y) G T}, and if Q and Ti are any operators from X to Y, we let 

g + n = {{x,y + z) I (x,y) G Q, (x,z) G 71}. 

An operator T: X ^ X is said to be firmly nonexpansive if {x — y,u — v) > ]]u — u]]^ for 
(x,ri), (y, u) G T. Moreover, T is said to be nonexpansive if \\Tu — Tv\\ < ]]x — y||. By 
[2, Proposition 5.14] we have the following result for a nonexpansive operator. 

Lemma 2.1 Let T : X —)■ X &e a nonexpansive operator with FixT A 0? {f^k} ® 

sequence in [0,1] satisfying Lk{7 — yk) = +oo. Let {w^} be generated by 

+ yk{Tw^ - w’^) withrx^GX. (4) 

Then, the sequences {w^} and {Tw^} converge to a point in FixT and for any w G FixT, 

\\w’^+^-wf <\\w’^-w\\^ - Hkil-tkk)\\Tw’^-w^W^ for all A; G N. 
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Next we recall from the monograph [2] the concept of the a-averaged operator. 

Definition 2.1 Let D be a nonempty subset of\, T: D be a nonexpansive operator, 
and a G (0,1) be a constant. Then the operator T is said to be a-averaged if there exists 
a nonexpansive operator TZ: D such that T = (1 — a)Id + aTZ. 

By [2, Prop. 5.15] we have the following result for an a-averaged operator T: X ^ X. 

Lemma 2.2 Let T-.H^Hbean a-averaged operator of a G (0,1) with FixT ^ 0, and 
{h-k} ^ [0, be a sequenee satisfying ~ Tk) = +oo. Let {w^} be generated 

by (4). Then, {w^} and {Tw^} converge to a point in FixT and for any w G FixT, 

-wf -wf - pLk{a-^ - pLk)\\Tw^ for all kGN. 

The following definition is about the metric subregular [17] of T at fx,y) G gphT. 

Definition 2.2 An operator T: X ^ Y is metrieally subregular at (x,y) G gphT with 
constant n > 0 if there exists a neighborhood U of x such that 

dist(x, T~^(y)) < Kdist(y, Tx) for all x gU. 

3 Linear convergence of several splitting algorithms 

In the first four subsection, we derive the linear convergence rate of the generalized PPA 
with both B and C vanishing, the over-relaxed FBS and the generalized DRS algorithm for 
problem (1) with the corresponding B and C vanishing, and Davis-Yin’s three-operator 
splitting algorithm for problem (1) under assumption that T is metric subregular at a 
point (z*,0) G gphT. In the last subsection, we will discuss the equivalence on metric 
subregularity condition between the inclusion operator T in (1) and fixed point operator 
in [29] for generalized PPA and over-relaxed FBS algorithm. Some sufficient conditions 
are also given to ensure the metric subregular of T at (2:*,0) G gphT. 

3.1 Linear convergence of generalized PPA 

The generalized PPA [19] for problem (1) with B and C vanishing takes the iteration step 

+ A/,(J.yjrz''- z^) for some 7 >0. (5) 

When Afc = 1, the iteration (5) reduces to that of the classical PPA [31, 36]. The 
linear convergence rate of the classical PPA is first established in [36] under the Lipschitz 
continuity of T~^ at 0. Later, Artacho et al. [1] and Leventhal [28] derived the linear 
convergence rate of the classical PPA under the metric regularity of T at a point (z, 0) G 
gphT and the metric subregularity of T at a point (z, 0) G gphT, respectively. Besides, 
the latter is weaker than the metric regularity of T at a point (z, 0) G gphT and the 
Lipschitz continuity of T“^ near 0 in the sense of [36]. We next establish the linear 
convergence rate of the generalized PPA under the same assumption as in [28]. 
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Theorem 3.1 Assume that the operator T is maximally monotone. Let he given 
by the generalized PPA with {Afc} C [0,2] satisfying — ^k) = +oo. Then, 

(a) the sequence {z^} converges to a point z* G and moreover, it holds that 

||_^fc+i _^||2 < yz G and VA: G N. ( 6 ) 

(b) If in addition T is metrically subregular at ( 2 ;*, 0 ) G gphJ^ with constant k > 0, then 

there exists fc G N such that 

dist(z^’''^, J'“^( 0 )) < 1 — — ^ dist(z^, J'~^( 0 )) for k> k. 

Proof: (a) Since iP is maximally monotone, jTiyj- is firmly nonexpansive (see [37, Theorem 
12.12]), and so is (l/2)-averaged by [2, Remark 4.24]. The result follows by Lemma 2.2. 

(b) Let = J^rz^. Notice that z* = J-yj^z*. We have jjx^ — 2 ;*|| < \\z^ — z*\\ by the 
nonexpansiveness of TTyj-, which by part (a) implies that ^ z*. Since P is metrically 
subregular at {z*, 0) G gph with constant k > 0, there exists A: G N such that 

dist(x^, J’“^( 0 )) < Kdist( 0 , J'(x^)) < —for all k>k, 

7 

where the last inequality is due to 'y~^{z^ — x^) G Px^ implied by x^ = J.yj^z^. Then, 

dist(.^^.T-l(0)) <dist(x^J^-^(0)) + |]/-x^|] < (l + -)||z^-x^|| for A:>L 

7 

Combining the last inequality with inequality ( 6 ) yields that for all k > k, 

d\stiz’^+\p-\0)f < -n^-i( 0 )(/)lP 

< ||z" - n^-i(0)(z")|p - Afc(2 - Afc)||z" - x'^f 

= dist(z^7-l(0))2 - Afc (2 - Aa,)1|z'= - 

dist( 2 ;^, 

where n_;r-i(Q)(-) is the projection operator onto The proof is completed. □ 

It is worthwhile to point out that Corman and Yuan [13] derived the linear conver¬ 
gence rate of the generalized PPA under the strong monotonicity of P, which is more 
stringent than the metric subregularity of P at (x,y) G gphJ^. We also observe that 
Liang et al. [29] establish the similar local linear convergence rate under the metric 
subregularity of (Z — TTyj') at (^*, 0) G gph (Z — By the Lemma 3.3 in subsection 

3.5, we will show that this condition is equivalent to the metric subregularity of P at 
(z*,0) G gphJ^. Very recently, Tao and Yuan [39] also established the linear convergence 
rate of the generalized PPA under the Lipschitz continuity of P~^ near 0 in the sense of 
[36], which is stronger than the metric subregularity of P at ( 2 ;, 0 ) G gphJ^. 


Afc(2 — Afc)7' 
(7 + 
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3.2 Over-relaxed forward-backward splitting algorithm 

The over-relaxed FBS algorithm for (1) with B vanishing takes the iteration steps: 

which takes the form of equation (4) with T := — 7 C) where 7 > 0 is the stepsize. 

When Afc = 1, the iteration (7) reduces to the FBS algorithm studied in [22, 34, 11]. 

For the sequence generated by (7), we have the following linear convergence 

result under the metric subregularity of T" := -|- C at a point ( 2 :*, 0 ) G gphT”. 

Theorem 3.2 Let B be P-cocoercive and { 2 ;^} be the sequence generated by (7) with 
7 €( 0 , 2 / 3 ) and Xk(^[0,6] such that Y^^=oXk{d—\k) =+ 00 , where (5 = min(l, Then, 

(a) the sequences { 2 :^} and {'Tz^} converge to a point z* G FixT = T'“^( 0 ), and 

\\z’^+^ - zf < \\z’^ - zf - Xk{S - Xk)\\Tz^ - z^f Vz G J-^(O) and Vh G N. ( 8 ) 


(b) If in addition IF is metrically subregular at the point ( 2 :*, 0 ) G gphT" with constant 
K > 0, then there exists A: G N such that for all k > k, 


dist(z''+\j^"H0)) < 



- Afc) 
(7 -I- k)2 


dist( 2 ;^,T' ^( 0 )). 


Proof: (a) From the proof of [2, Theorem 25.8], it follows that T is (l/h)-averaged. 
Thus, the result of part (a) follows directly from Lemma 2.2. 

(b) Let = Fz^. From the dehnition of T and the single-valuedness of C, we have 
z^ — jCz’^ e + 'jAx^. Hence, 7 “^ ( 2 :^ — x^) + Cx^ — Cz^ G Ax^ + Cx^ = Fx^. In 
addition, from part (a) it follows that —>■ 2 ;*. Now by the metric subregularity of F 
at ( 2 ;*, 0 ) G gphJ-”, there exists A: G N such that for k > k, 

dist(x^, J’“^( 0 )) < Kdist( 0 ,/'(x^)) < k]| 7“^(2;^—x^)-|-— C2:^|| 


< K 


/y ^||2:^ — x*^P -b (1- —) \\Cx^ — Cz^W^ < (k/7)||x^ — z^ 


where the third inequality is using the cocoercivity of C, and the last one is due to 
7 G (0,2/3). The last inequality immediately implies that for k > k, 

dist(z^,J^-pO)) < dist(x'=,J^-pO)) + ]]z^-x'^ll < {l + k/j)\\x^ - z^\\. (9) 


Combining inequality (9) with inequality ( 8 ), we obtain that for all k > k, 

dlstiz^^\F-\0)f < -n^-i(0)(zp|f 

< 11^"= - n^-i(0)(/)f - Xk{s - Xk^z^^ - X 
l^Xk{d-Xk) 


k\\2 


< 


1 - 


(7 -b kY 


dist(2;^,J' ^(0))^. 
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This implies the desired result of part (b). The proof is then completed. □ 

By Theorem 3.2, one may see that the linear convergence rate coefficient is smallest 
when Afc = ^6. Recall that Chen and Rockafellar [11] derived the linear convergence 
of the FBS algorithm under the strong monotonicity of := A + C, which implies the 
single-valuedness and Lipschitz continuity of and then the metric subregularity of 

at € gphJ^. Notice that the linear convergence is also derived in the work of 

Liang et al. [29] under the metric suregular of (X—T) at the point G gph(X—7”). 

In the following lemma 3.3 in subsection 3.5, we show that the metric subregularity of 
(Z—T) is equivalent to the one of X” at (^*,0) G gph(X'). 


3.3 Generalized Douglas-Rachford splitting algorithm 

Given 7>0, the generalized DRS method for (1) with C vanishing takes the iterations: 

' = J^Bx\ 

< y^ = J,j,{2z^-x^), ( 10 ) 

which can be rewritten as the form of (4) with T := ^((2j7iyyt +^)) he., 

= x^ + Afc(rx^-x^). (11) 

When Afc = 1, equation (10) gives the DRS method [30, 18], and when A^ = 2 it gives 
the PRS method [30, 35]. Before stating the linear convergence rate of the generalized 
DRS method, we establish the relationship between the set J-'“^(0) and the set FixT. 

Lemma 3.1 The set J-'“^(0) has the following relations with the fixed-point set FixT.’ 

(a) T“^(0) = X 7 B(FixT). If B is single-valued, FixT ={T 

(b) FixT C (T—74.)(T“^(0)). If A is single-valued, then FixT = (T— 7 Al)(T“^( 0 )). 

The proof of the above lemma is given in the Appendix. Next, we show the DRS 
method converges linearly under the metric subregularity of T at ( 2 ;*, 0) G gphT. 

Theorem 3.3 Let {x^}, {y^} and {z^} be generated by the generalized DRS method with 
{Afc} C [0,2] such that ~ ^k) = + 00 . Then, the following statements hold. 

(a) and {z^} converge to z* G T~^(0), and {x^} converges to x* G FixT and 

||xfc+i-x]]2 < ||x^-x]]2-Afc(2-Afc)||Tx^-x*^f Vx G FixT and V/t G N. (12) 


(b) If A is single-valued and Lipschitz continuous with modulus and T is metrically 
subregular at (2;*,0) G gphT with constant k > 0, then there exists A: G N such that 


dist(x*^+\FixT)< Jl- 


Afc(2 —Afc) 


[2-hv^l-h7^/3 ^(l + «:(7 ^))]' 


■ dist(x^,FixT), \/k>k. 
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(c) IfB is single-valued and Lipschitz continuous with modulus and T is metric 
subregular at (z*,0) G gphJ^ with constant k > 0, then there exists A: € N such that 


dist(x*^+SFixr) < Jl - 


l3^Xk{2 - Afc) 


(7 + / 3 ) 2(1 + Ka/7 2 + ^ 2y 


■dist(x^,FixT), Vk > k. 


Proof: (a) It is easy to check that T is nonexpansive. The result follows directly from 
Lemma 2.1 and the first equality of Lemma 3.1(a). 

(b) By the iteration steps (10), it follows that {x^—z^) G Bz^, {z^—x^-\-z^^^) = Ay^ 

and 7 “^(z^ —y^) G Ay^-\-Bz^. This, along with the single-valuedness of A, means that 

7-^(z^ - y^) + Az'^ - Ay'^ G Az'^ + Bz^ = Tz^. 

By part (a), using the metric subregularity of T at the point (2:*,0) and the Lipschitz 
continuity of A, it follows that there exists A: G N such that for all A: > A:, 

dist( 2 ;^,^”“^(0)) < Kdist(0, Tz^) < K\\'y~^{z^ — y^) + Az^ — Ay^\\ 

<^(7-i+/3-1)||/_/||, 

which further implies that 

dist(y^, J'"^(0)) < dist(^^,7'"^(0)) + ||/ - y^\\ < [l + k( 7 “^+/ 3 “^)] ||y^ - z^\\. (13) 

Let # = nj--i(o)(y^) — 7Al(nj--i(o)(y^)). From Lemma 3.1(b), # G FixT. In addition, 
notice that 2z^ — = y^ + '^Ay^ by the second equality in (10). Thus, for all k > k, 

dist(x^Fixr) < llx"^ - n^-i( 0 )(y") +7^(n^-i(0)(/))ll 

= 112^"= -y’^- 7^ - n^-i( 0 )(/) + 7Al(n^-i(0)(/))ll 

< ||2z" - 2y"|| + ||y" - n^-i(o)(/) - 7^1/ + 7Al(n^-i(o)(/))|| 

< 2||^'= - y^ll + v^l + 72/3-2dist(y^,J^-^(0)) 

< [2 +v^l + 72 / 3 - 2(1 + k(7“^+/3“^))] \\z^ - y^ll, 

where the third inequality is using the Lipschitz continuity and monotonicity of A, and 
the last one is due to (13). In addition, from equation (11) and the last equality of (10), 
we have Tx^ — x^ = —— ' = (y^ — z^), which together with (12) implies that 

dist(x^+\Fixr)^ < \\x'^+^ -UFi^T{x^)f 

< Wx’^ - - Afc(2 - Afc)||rx'^ - x’^f 

= dist(x^, FixT)^ — Afc(2 — Afc)||Tx^ — x^||^ 

= dist(x^Fixr)2 - Afc(2 - Afc)||y'^ - (14) 

The desired result of part (b) follows directly from the last two inequalities. 
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(c) Notice that B is single-valued and Lipschitzian. Thanks to part(a) and (b), we have 

- /) + - Bz^ e + By'^ = Ty^ 

From the metric subregularity of T at the point (z*,0), there exists k > k such that 

dist(/,jr-^(0)) < Kdist(0,-F/) < K\\-f-\z’‘ - y^) + By’^ - Bz^\\ 

< — y^W for k > k, 

where the last inequality is using the monotonicity of B. Consequently, for all k > k, 

dist(z^7-^(0)) <dist(/,^-i(0)) + ||2'^-/|| < [i + KVi-^+p-^]\\y^ -z^l 

Let z^ = B{Jijr~i(^Q'^{z^)). By Lemma 3.1(a), clearly, nj--i(-o)(2;^) -t- 72 ^ € FixT. More¬ 
over, using the Lipschitzian property of B, we have \\z^ —Bz^\\ < ;0“^||njr-i(o)(2^) — 

In addition, according to = z^ + jBz^ G further obtain 

dist(x^,FixT) < ||x^ — njr-i(o)(. 2 ^) — 72^11 

<||z^-n^-i(o)(z^)||+7p"-^"ll 

<(l + 7/3-')lk"-n^-i(0)(^")ll 

< (1 + 7 / 3 -^) [1 + KV 7 - 2 +/ 3 - 2 ] 11 / - z^W iovk>k. 

Combining this inequality with (14) yields the desired result. The proof is completed. □ 

Remark 3.1 Giselsson [23] and Davis et al. [15] recently derived the linear convergence 
rate of the generalized DRS method under the assumption that A is strongly monotone and 
B is fi-Lipschitz continuous, which is stronger than the assumption of Theorem 3.3(c). 

It is well known that the generalized DRS method is a generalized PPA associated 
with operator in the sense that T = {X+ab ^1" Theorem 5]. 

That is, the sequence {x^} in (10) can be generated by the following iteration step 

x^+i = x" + Afc(J5„^,^x'=-x'^). (15) 

By Theorem 3.1, we also have the linear convergence rate of the generalized DRS method 
under the metric subregularity of at (x, 0) € gphiS-y^^^g, stated as follows. 

Theorem 3.4 Let {x^} he the sequence generated by equation (15) with {Afc} C [0,2] 
and -^^(2 — A^) = -|-cx). Then, the following statements hold. 

(a) {z^} and {x^} converge to z* G T~^(0) and x* G respectively, and 

||x^+^-x||^ < ||x''-x|p-Afc(2-Afc)||JT^^gx'^-x^lP Vx G and Vk G N. 
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(b) If is metrically subregular at (x*,0) with constant k > 0, then there exists 

A; G N such that for all k > k, 


dist(x^+^ 





Afc(2 — Afc) 

(1 + 


dist(x^,5^j^g(0)). 


(16) 


Corman and Yuan [13] derived the linear convergence rate of the generalized DRS 
method under the assumption that is strongly monotone (implied by the strong 

monotonicity of T and one of A and B is firmly nonexpansive), which is stronger than 
the metric subregularity of by Proposition 3.2 in subsection 3.5. More recently, 

Liang et al. [29] establish its local linear convergence rate like (16) under the metric 
subregularity of (Z — T) at a point (z*, 0) G gph (Z—T) which is equivalent to the metric 
subregular of at (^*,0) G gph5..y^^^e according to lemma 3.3 in the subsection 3.5. 

Although, the linear convergence of the generalized DRS algorithm can be deriveed 
under the metric subregularity of or (X — T) at a point of its graph, this regular 

condition may be too difficult to be certified since that is highly compound of A and 

B. On the contrast, the metric subregularity of Z” := A + B at the point {z*, 0) G gphZ" 
may be slightly easier to check due to its simple formulation. In the last subsection, we 
will give some sufficient conditions to ensure the metric subregularity of Z” = A + B. 


3.4 Davis’ three-operator splitting algorithm 

Davis’s splitting method [16] for the inclusion problem (1) takes the following iterations 

< y^ = J^A^z^-x'^-^Cz^), (17) 

^x’^+^ = x^ + Xk{y^-z>^). 

Let T := Z — J^B + J'^A ° — Z — yC o J^b)■ Then, with this operator, the iterations 

in equation (17) can be compactly written as x^~^^ = x^ + Xki'Tx^ — x^). 

The following lemma present the relation between the solution set Z'“^(0) and the 
fixed-point set FixT. For the sake of coherence, its proof is given in the appendix. 

Lemma 3.2 The set Z'“^(0) has the following relations with the fixed-point set FixT: 

(a) T~^(0) = 77yg(FixT). If B is single-valued, FixT = (Z-|- 7 ;B)(T“^( 0 )). 

(b) FixTZ (Z—7(M+C))(T“^(0)). If A is single-valued, FixT= (Z—7(M+C))(T~^(0)). 

Theorem 3.5 Let {x^} be the sequence generated by (17) with {A^} C [0, such 

that ~ ^k) = +00 for 7 G (0, 2tI). Then, the following statements hold. 

(a) {y^} and {z^} converge to z* G T“^(0); and {x^} converges to x* G FixT, and 

llx^"''^ —x]|^ < ]|x^—x|p — Afc( ^^ —Afc) ||Tx^ — x^lp Vx G FixT and VA: G N. 

^ 1 / 
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(b) If A is single-valued and Lipschitz continuous with modulus 1//3 and T is metric 
subregular with constant k > 0 at (z*,0), then there exists A; G N such that 


FixT) < (1 — ^) dist^(x^, FixT) for k > k (18) 


with 

_Afc(4^-7-2gAfc)_ 

2'i9[(2 + + ( 71 ?-^ +Y^l + 72 ^- 2 )(i + K(^-i-(-^-i))j^ 


(c) If B is single-valued and Lipschitz continuous with modulus 1//3 and IF is metric 
subregular with constant k > 0 at (2;*,0), then there exists fc G N such that (18) 
holds for all k > k with 


Q = 


Afc(4^9-7-20Afc) 


2^9(1+ 7/3 1) 1 + k(A + A Jl+ max 



Proof: (a) By [16, Proposition 3.1], F is a-averaged with a. = for 7 G (0,27?). By 
Lemma 3.2(a), J^“^(0) = ^i7iye(FixT). Thus, the result directly follows by Lemma 2.2. 

(b) From the iteration step (17), — G Bz^ axi(i^~^{2z^—x^— ^Cz^—y^) G Ay^. 

Hence, we have 7 “^(z^ — y^) G Ay^ + Bz^ + which further implies that 

- y^) + Az^ - Ay^ G Az^ + Bz^ + Cz^ G Fz^. 

Since F is metrically subregular at (^;*, 0) with constant k and z^ —>■ z* by part (a), there 
exists A: G N such that for all A: > A:, the latter inequalities hold 

dist(z^J^-^(0)) <Kdist(0,.F(z^))<At||7-i(z^-/)+^/-^/||<K(7-^+/3-i)||z^-/|| 


where the last inequality is due to the Lipschitz continuity of A. Thus, for all k > k, 


dist{y’",F ^(0)) < dist(z'',7' ^(0)) + Hz'" - //'‘|| < 1 + ^(7 ^+/3 ^) 


T7 —1 


-y 


(19) 


Let z^ = nj--i(Q)(y^) — 7 (^ + C)(nj--i(Q)(?/^)). From Lemma 3.2(b), z^ G FixT. In 
addition, notice that x^ = 2z^ — y^ — jAy^ — jCz^ by (17). Hence, for all k > k, 

dist(x^Fixr) < ||x'=-n^-i(0)(/) + 7(Ai + c)(n^-i(0)(/))|| 

= || 2 z" - / - 771 / - ^Cz’^ - n^-i( 0 )(/) + 7(AI + C)(n^- 1 ( 0 )(/))!! 

< 112 / - 2/11 + 11 / - n^-i( 0 )(/) - 7-4/ + 7-4(n^-i(0)(/))|| 

+ 1 I 7 C/ - 7 C/ + 7 C(n^-i( 0 )(/)) - 7 C/II 

< (2 + 7^9“^)||/ - /II + + /r+W^)dist(/, J^-^O)) 

< [(2 + 71?“^) + (7i9“^ + /1 + 72/3-2)(1 + k{j~^+(I~^))] ||/ - /||, 
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where the third inequality is using the monotonicity of A and the last one is due to (19). 
In addition, using part (a) and the same arguments as for Theorem 3.3(b) yields 

dist(x^+\FixT)^ < dist(x^,FixT)^ - ^ -Afc)||/ - z^f. (20) 

The desired result then follows from the last two inequalities. 

(c) By the proof of part (b), we have —y^) € Ay^ + Bz^ + Cz^. This along with the 

single-value IS yield — y^) + {By^ — Bz^) G Ay^ + By^ + Cz^. By the cocoercivity 

of C, we obtain that — y'^) + {ri^ —^^) + Cy^ — Cz^ G Ay^ + By^ + Cy^ = Ty^. Since 

T is metrically subregular at (z*,0) with constant At, from part (a) and the Lipschitzian 
of B and the cocoercivity of C, it follows that there exists k such that for all k > k, 


dist(/,^-i(0)) < Atdist(0,7-(/)) < At||7-'(^" - /) + (r?" - e)+Cy^ - Cz 


-1/ 


< K 

< K 




/3 


+ \ — + max ( 


7 


7 — 

71^2 


> 0 ) 


1^-/1 


Combing this inequality with dist(z^, J” ^(0)) <dist(?/^, J” ^(0))-|-||z*^— 


dist(2:^, ^(0)) < 






implies that 
,\/k>k 


From FixT = (0), it follows that nj--i(Q)(2:^)-|-7;B(nj--i(o)(2:^)) G FixT. More¬ 

over, using the Lipschitzian of S yields \\B{Iljr-i(^Q-j{z^)) — Bz^\\ < l5~^\\Iijr-i^Q'^{z^) —z^\\. 
Combining this inequality with the facts that = z^ + 'jBz^ and njr-i(Q)(. 2 ^) —>■ z*, 
and using the same arguments as for those of Theorem 3.3(c) yield that for all fc > fc. 


dist(x^,Fixr) < (1 + 7/3"^) 


1 + K 


11 , , 
— -I- -\ 1 -I- max 
/3 7 


7^ — 271? 


.»)) 



Combining this inequality with (20) yields the desired result. The proof is completed. □ 

Davis and Yin [16] derived the linear convergence rate of their algorithm under the 
condition that one of A, B and C is strongly monotone and one of A and B is single-valued 
and Lipschitz continuous, which is stronger than that of Theorem 3.5 (b) and (c). 


3.5 Sufficient conditions for the metric subregularity 

In this section, we give some sufficient conditions to ensure the metric subregularity of the 
maximal monotone operator F -.= A + B under the condition that B is single valued and 
Lipshcitz continuous with modulus Write ^{z) = z — — ''yB)z which is clearly 

a single valued mapping. In the following lemma, we give an equivalent characterization 
on the metrically subregularity of F at z* for 0 G F{z*). 
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Lemma 3.3 Let T := A + B where A is maximal monotone and B is single valued and 
Lipschitz continuous with modulus Then, the mapping R is metrically subregular at z* 
for 0 G 7Z(z*) if and only if the operator F is metrically subregular at z* for 0 G F(z*). 

Proof: By the metric subregularity of R at z* for 0 G R(z*), there exist a constant 
k' > 0 and a sufficiently small 5' > 0 such that 

dist(z,7^■^(0)) < K'||7e(^)||, VzGB(2*,(5') ( 21 ) 

where ]B(z*, <5') denotes the closed ball in the space El centered at z* with radius S'. Let z 
be arbitrary point from ]B(z*, 5'). Take yGF(z) as the point such that dist(0, F{z)) = ||y||. 
Notice that F = A + B and B is single valued and lipschitz continuous, it is easy to get 
that z + 7 ?/ — yBz G {Z+'^A)z which in turn implies that z = J^a{z + 72/ — 'yBz). 
Together with the last equation and the metric subregularity (21) of TZ, we obtain that 

dist( 2 ;, 7 '"^( 0 )) =dist( 2 :, 7l“^(0)) <K\\TZ{z)\\ = K'\\J.yji^{z + jy- 'yBz) - J.yA{z - yBz)\\. 

Notice that ZfyA is nonexpansive. This along with the equality dist(0, J^(z)) = || 2 /|| yield 

dist(2;, J'“^(0)) < K'7dist(0, J'(z)), 

which shows that the operator F is metrically subregular at z* for 0 G F(z*}. 

Conversely, suppose that F is metrically subregular at z* for 0 G F{z*). Then, there 
exist a constant k > 0 and a sufficiently small (^ > 0 such that 

dist(z, J-^(O)) < Kdist(0, J'(z)) Vz G B(2*,(5). (22) 

Take S' = ■ Notice that the equation holds Jly_4(X—70)z* = z* since G J^“^( 0 ). 

Then, ^ B( 2 :*, 5 ) due to ||— 7 . 8 ) 2 ; — 2 :*|| < {l + 'yl3~^)\\z — z*\\ <S. 

Combine 2 ;— 7 .B 2 ; —Jly^( 2 ;— 7 .S 2 ;) G 7 ^(jly^( 2 :— 7 .S 2 :)) and metric subregularity of F yield 

dist(2;, 7 ^“^( 0 )) = dist(2:, J^“^(0)) 

< dist(l7iyyi(2;-702;), J'"^( 0 )) + ||2; - J.yAiz - 'yBz)\\ 

< Kdist( 0 , J'(Jly^(2; - 'yBz))) + ||2; - J.^a{z - yBz)\\ 

= Kdist( - B{J.yA{z - yBz)),A{J.yA{z - l^z))) + ||2; - J.^a{z - yBz)\\ 

< k|| - B{J^a{z - yBz)) - y~\z - J.^a{z - yBz)] + Bz)\\ + \\z - J^a{z - l^z)\\ 

< {l + y~^K +I3 ~^k)\\z - J^a{z- yBz)\\ = (1 + 7"^k +/3"^^^)||7^(2;)|| 

which implies TZ is metric subregularity at (2;*, 0 ) GgphT^. The proof is completed. □ 

By the above lemma and Theorem 3.3, the generalized DRS algorithm is linear con¬ 
vergence if TZ is metric subregular at 2;* for 0 G TZ{z*). Moreover, When B is reduced 
to .8 = 0, strengthened as a cocoercive operator and specified as B = C + P with C 
being cocoercive and T) being single valued and lipshcitz, respectively. By Theorem 3.1 
3.2, 3.5 and above Lemma 3.3, we know that the generalized PPA, the over-relaxed FBS 
algorithm, and the Davis-Yin’s three operator splitting method are linearly convergent 
with TZ is metric subregular at 2;* for 0 G TZ{z*) accordingly. Next, we give a sufficient 
condition to ensure the metric subregularirty of TZ at z* for 0 g 7 ^( 2 ;*). 
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Lemma 3.4 The single-valued mapping R is metrically subregular at z* for 0 G TZ{z*) 
if the following projection type error bound [fO, Eq. 5] holds 

dist(z,7^“^(0)) < k!'\\TZ{z)\\, V z with ||7^(^)|| < 5” (23) 

Proof: Let 5' = z(zM{z*,5'), using the equality z* = yields 

||7^(^)|| = \\z-z*-{J-,A{'l^--fB)z-J-,A{'I-^B)z*)\\ < {2 + ^r^)\\z-z*\\ < 5" (24) 

Together with the above inequality and the projective bound (3.4), we get the desired 
results that R is metrically subregular at 2 * for 0 G TZ{z*). □ 

Next, we give certain instances with the projection type error bound (23) or metric 
subregularity of TZ holding. Consequently, the metrically subregularity of J- =A+B holds 
at 2 ;* for 0 G J-{z*). The proof of the following proposition is followed directly according 
to [40, 43, 42]. Here, we omit the details. 

Proposition 3.1 Let T := A + B where A is maximal monotone and B is single valued 
and Lipschitz continuous. Then, the operator T is metrically subregular at a point z* 
for 0 G J-(z*), i.e., there exists k, 6 such that inequality (22) holds whenever one of the 
following statements holds. 

(Cl) J- := A B is strongly monotone; 

(C2) B is affine operator and A is polyhedron operator; 

(C3) B = TV/(T') + C where f is strongly convex and gradient Lipschitz £ is linear 
operator and C is a constant. A is the subdifferential operator of ip norm with 
pG [l,2]U{oo} or polyhedral convex function; 

(C4) B = £'Vf{£-)-\-C where f is strongly convex and gradient Lipschitz and £ is linear 
operator and C is a constant. A is the subdifferential operator of the nuclear norm. 
In addition, —Bx G ri(9||x||*). 

To end this subsection, we make some comments on the metric subregularity of 
S'y,A,B at a point x* with 0 G S-^^a,b{x*). Up to now, we are not clear whether the 
metric subregularity of S-i,A,B at (x*,0) G gph(5.^^ A,b) is weaker than that of J-:=A+B 
at (2:*,0) G gphT” or not when A or B is single-valued and Lipschitz continuous. The 
following proposition gives a sufficient condition to guarantee the metric subregularity of 
<S'y,A,B at a point (x*,0) G gph (^.y^^q^g). Its proof is also provided in the appendix. 

Proposition 3.2 If IF is strongly monotone with constant a > 0 and one of A and B is 
single-valued and Lipschitz continuous with modulus /3 > 0, then 

for w (25) 

This implies that S-f^A,B is metrically subregular at (x*,0) G gph(5y^^^e). 
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4 Toy examples 

In this section, we first consider the following nonsmooth convex optimization problems 


min f{x) + g{T>x) 


(26) 


where /:Z —>■ [— 00 , + 00 ] and g:Y ^ [~oo, + 00 ] are low semicontinuous convex function, 
and D : Z —7> Y is linear operator. Notice that the problem (26) embodies an abundance of 
popular applications such as famous Rudin-Osher-Fatemi (ROF) denoising model [38], 
TVLi minimization model [7], the convex image segmentation model [8, 24] and the 
.£i/.£i-regularization model [9]. It is obvious that any optimal solution of (26) satishes 
the inclusion 0 G df{x) + 'D*dg(T>x). Involved in the dual variable y G Y, the above 
inclusion can be reformulated as the inclusion (0,0) G (71 + T 2 ){x,y) with 


/ Ti{x,y) = {V*y,-Vx) 

1 '7i(x,y) = {df{x),dg*{y)) 


It is obvious that 71 is single valued, Lipschitz continuous and affine operator. Hence, we 
can apply the generalized DRS algorithm for the inclusion (27) with following iterations 


' x\ = {l + {zf - 7 'D*z^) (28a) 

yf = {1 + -f‘^VV*)-\-iVz\ + 4) (28b) 

4 = min f{x 2 ) + lla ;2 - (2xi - 4)f /( 27 ) (28c) 

4 = min 54 ^ 2 ) + ||y 2 - (2^ - 4)llV(27) (28d) 

4+1 = 4 + Afc(x^ - x^) (28e) 

,4+i = 4 + A44-4) (28f) 


with 7 > 0 and relaxation parameter Afc G (0, 2). In the following, we specify model (26) 
as the .^i/^i-regularization model proposed by Chan et al. [9] with the following form 


min ||Hx — 6||i + A||x]|i (29) 

X 

min \\Ax — b\\^ + A||x]|i 

X 

Now, we apply the above generalized DRS algorithm (28a)-(28f) to the above li/li 
regularization minimization (29). In this case, we get that 7i{x,y) = {A^y,h — Ax) is 
singled valued and Lipschitz continuous operator and 72{x,y) = (5||ic]]i, Al||y||^<i(y)) is 
polyhedral operator. By condition (C2) in Proposition 3.1, we know that the algorithm 
(28a)-(28f) converges linearly when it is applied to the ii/ii regularization minimization 
(29). In the following, we verify this linear convergence result by the £i/.£i-regularization 
minimization with random generated sensing matrix A and regularization parameter 
A = 1. The Figure 1 shows that numerical performance of generalized DRS algorithm 
(28a)-(28f) when it is applied to the problem (29), which is coincided with Theorem 3.3 
that the prime-dual points sequences {{x^,y^)} and sequences {(zi, 4)}converge linearly. 
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Figure 1: £i/£i-regularizatioii minimization Figure 2: ^p-regularization minimization 


Next, we consider the ip norm regularization problem with p G [1,2] |J{+oo}. 

min /{Vx) + g{x) (30) 

where / :¥—)■ [—oo, +oo] is a strongly convex and gradient Lipschitz continuous function 
and H : X —>■ Y is a linear operator, and g{x) = '^jizjWj^xj^p where J \s a non¬ 
overlapping divisibility of index set {1,2,-•• ,n} and wj > 0, and ||xj||p denotes the 

Ip norm defined by ||xj||p = ■ This problem (30) incorporates massive 

applications such as Group-lasso regularization [41], ^i^p-regularization regression [21, 20] 
and the referees in [42], Here, we consider the following ip norm regularization problem 

min - 6]]^ -I- Ajjxllp (31) 

Now, we apply the over-relaxed forward backward splitting algorithm (7) and the gen¬ 
eralized Douglas Rachford splitting algorithm (10) to the problem (31) with random 
generated sensing matrix A, regularized parameter A = 1 and p = 2, respectively. The 
figure 2 shows performance of over-relaxed FBS and generalized DRS algorithm when 
they are applied to the toy example (31), which is coincided with Theorem 3.2, 3.3. 

5 Conclusion 

In this paper, for the inclusion problem (1), we have established the linear convergence 
rate of the generalized PPA and several popular splitting algorithms under the metric 
subregularity of the composite operator, which is much weaker than the existing ones that 
almost all require the strong monotonicity of the composite operator. Some sufficient 
condition are provided to ensure the metric subregularity of the composite operator 
holds at a point (z*,0) G gphJ^. The preliminary numerical performances also support 
the theoretical results. 
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Appendix 

Proof of Lemma 3.1: Part (a) directly follows from [2, Proposition 25.1(ii)]. We next 
make use of in [19] to prove the inclusion of part (b), where is defined by 

= I + 7^5 u — v) \ {u,b) ^ B, {v, a) ^ A, V + = u — . (32) 

From [19, Theorem 5], it follows that FixT = S~\ g(0) = IJ^gx \_^ + "i{—Au fl Bu)\. Let 
X be an arbitrary point from FixT. Then, there exist u G X and b G {—Au H Bu) such 
that X = tt + 76 G (T— 'yA){u). Clearly, 0 G Au + Bu = Tu, i.e., u G T~^(0). Hence, 

^ e U.g^-i(o)(^ - 7^)^ = (^ - 7 A)(T-Ho)). 

The inclusion then follows from the arbitrariness of x in the set FixT. 

Now assume that A is single-valued. To establish the equality, it suffices to argue 
that (T — 7 A)(T~^( 0 )) C FixT. Let x be an arbitrary point from (T — 7 A)(T“^( 0 )). 
Then there exist z G T~^(0) and y = Az such that x = z — yy. Since 0 G Az + Bz, we 
have — Az = —y G Bz. Thus, x G z + ^Bz, which by z G T“^(0) and part (a) implies that 
X G FixT. The inclusion follows by the arbitrariness of x in (T— 7 A)(T“^( 0 )). Hence, the 
proof is completed. □ 


Proof of Lemma 3.2: Part (a) follows from [16, Lemma 3.2]. ft suffices to prove part 
(b). From [16, Lemma 3.2], it follows that FixT = Utiej'-po) \fi + — Cu) OBu)\. 

Let X be an arbitrary point from FixT. Then, there exist u G T~^(0) and b G {—Au — 
Cu) n Bu such that x = u + ^b £ ~ 7 (A + C)] u. This immediately implies that 

xG U [T-7(A + C)]2= [T-7(A + C)](T-'(0)). 

2eJ^-i(o) 

By the arbitrariness of x in FixT, the inclusion follows. Now assume that A is single¬ 
valued. We only need to argue that \X—'^{A+C)\ (T“^(0)) C FixT. Let x be an arbitrary 
point from [T — 7 ( 7 ! -|- C)] (T“^(0)). Then there exist G T“^(0) and y = {A + C)z 
such that X = z — 'yy. Since 0 G Az Bz + Cz, we have —{A + C)z = —y G Bz. Thus, 
X £ z + 'yBz. This, along with 2; G T“^(0) and part (a), implies that x £ FixT. By the 
arbitrariness of x in [T — j(A + C)] (T~^(0)), the inclusion follows. □ 


Proof of Proposition 3.2: Let x be an arbitrary point from dom<S.Y^^^e. Since 
is maximal monotone (see [19, Theorem 4]), the set Sj^a,b(x) is closed convex by [37, 
Exercise 12.8]. In the following, we proceed the arguments by two cases as shown below. 
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Case 1: ^ is single-valued and Lipschitz continuous with modulus /3 > 0. From 
the closed convexity of and the expression of there exist u,b,v G X with 

{u, b) € B and v + 'jAv = u — jb such that u — v = dist(0, S-y^_A^s{x)) for x = r; + 76 . So, 

'y~^{u — v) + {Au — Av) G Fu. (33) 

By Lemma 3.1(b) and the remark after it, we have 5“\e(0) = {T-lA){F~\^)). Thus, 

dist(x, = dist(u - ^Av, < Ik “ - z* + ^Az*\\ 

< ||tt — z*|| + 7 / 3 ||r; — 2*11 < (1 + 7 / 3 )|k — z*\\ + 'yI3\\u — v\\, (34) 

where is an arbitrary point from J^~^(0). Since F = ^ + .B is strongly monotone with 
constant a, F~^ is single-valued and Lipschitz continuous with modulus a~^, i.e., 

--^"^0)11 < VruGX. (35) 

Let w G Fu be such that ||u;|| = dist(0, J^u). Then, by the last inequality, it follows that 

||u — z*\\ = ||/'“^(lt;) — J'“^( 0 )|| < q;“^ 1 |u;|| = Q:“^dist( 0 , J'u). 

Combining this inequality with inequalities (33) and (34), we have that 

dist(x,5“^ g(0)) < a“^(l + 'yf3)dist{0,Fu) + 'yf3\\u - u|| 

< a“^(l + 7/3)||7“^(u — u) -|- {Au — w4'(;)|| -|- 7/3||u — uH 

< [a"kl + 7/3)(7“^ + /3) + 7/3] Ik - ^11 

= [a“kl + 7/3)(7“^ +/3) + 7/3]dist(0,5.^,^,i3(a;)). 

This, along with the arbitrariness of x in dom5..y^^^g, is equivalent to saying that 

for w eX. 

Notice that ‘S~^g(0) is singleton due to the strong monotonicity of F. Together with 
the last inclusion, is metrically subregular at (x*,0) G gph<S.Y^_ 4 ^g. 

Case 2: B is single-valued and Lipschitz continuous with modulus /3 > 0. From 
the closed convexity of S^^j^^s{x) and the expression of there exist rt, u, a G X with 

{v, a) G A and u + 7 a = u—'yBu such that u — v = dist( 0 , S^^_a^b{x)) for x = v+^Bu. So, 

7 “^(u — u) -|- {Bv — Bu) G Fv. (36) 

By Lemma 3.1(a) and the remark after it, we have ‘5~]^g(0) = (X-|- 7 B)(J^“^( 0 )). Thus, 

dist(x,5^3l,B(d)) = dist(u-b 7 'Su, 5 ^]^g( 0 )) < \\v +-^Bu - z* - Bz*\\ 

< ||u — z*|| -|- 7/3 ||u — 2*11 

< (1-I-7/3)||u - 2*11-I-7/3||u - u||, (37) 
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where z* is an arbitrary point from Since J- = A + B is strongly monotone 

with constant a, is single-valued and Lipschitz continuous with modulus a~^. Let 
w G J-v be such that l|u;|| = dist(0, Then, from (35) it follows that 

||u — z*\\ = — J^“^(0)|| < a“^||u;|| = Q:“^dist(0, 

Combining this inequality with inequalities (36) and (37), we have that 

dist(x,5“;J4 g(0)) < a“^(l + 7/3)dist(0,-Tu) -|- 7/3||tt - u|| 

< a~^{l + 7/3)||7“^(u — u) + {Bv — Bu)\\ + 7/3||ti — u|| 

< [a“^(l -b 7/5)(7"^ + P) + 7/3] Ik - ^^11 

= [a"^(l -|-7/3)(7"^ -1-/3) + 7/3]dist(0,5^,^,B(x)). 

The rest arguments are similar to those for Case 1, and we omit them. □ 
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